Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Primal-Dual Approach

Reinforcement learning is widely used in applications where one needs to perform sequential decisions while interacting with the environment. The problem becomes more challenging when decision requirement includes satisfying some safety constraints. mathematically formulated as constrained Markov process (CMDP). In literature, various algorithms are available solve CMDP problems a model-free manner achieve epsilon-optimal cumulative reward epsilon feasible policies. An epsilon-feasible policy implies that it suffers from constraint violation. important question here whether we can zero violations or not. To that, advocate use of randomized primal-dual approach and propose conservative stochastic algorithm (CSPDA) which shown exhibit O(1/epsilon^2) sample complexity violations. prior works, best for violation O(1/epsilon^5). Hence, proposed provides significant improvement compared state art.